37 research outputs found
LittleDarwin: a Feature-Rich and Extensible Mutation Testing Framework for Large and Complex Java Systems
Mutation testing is a well-studied method for increasing the quality of a
test suite. We designed LittleDarwin as a mutation testing framework able to
cope with large and complex Java software systems, while still being easily
extensible with new experimental components. LittleDarwin addresses two
existing problems in the domain of mutation testing: having a tool able to work
within an industrial setting, and yet, be open to extension for cutting edge
techniques provided by academia. LittleDarwin already offers higher-order
mutation, null type mutants, mutant sampling, manual mutation, and mutant
subsumption analysis. There is no tool today available with all these features
that is able to work with typical industrial software systems.Comment: Pre-proceedings of the 7th IPM International Conference on
Fundamentals of Software Engineerin
Measuring Coverage of Prolog Programs Using Mutation Testing
Testing is an important aspect in professional software development, both to
avoid and identify bugs as well as to increase maintainability. However,
increasing the number of tests beyond a reasonable amount hinders development
progress. To decide on the completeness of a test suite, many approaches to
assert test coverage have been suggested. Yet, frameworks for logic programs
remain scarce.
In this paper, we introduce a framework for Prolog programs measuring test
coverage using mutations. We elaborate the main ideas of mutation testing and
transfer them to logic programs. To do so, we discuss the usefulness of
different mutations in the context of Prolog and empirically evaluate them in a
new mutation testing framework on different examples.Comment: 16 pages, Accepted for presentation in WFLP 201
Class mutation operators for C++ object-oriented systems
Mutation testing is a fault injection testing technique around which a great variety of studies and tools for different programming languages have been developed. Nevertheless, the mutation testing research with respect to C++ is pending. This paper proposes a set of class mutation operators related to this language and its particular object-oriented (OO) features. In addition, an implementation technique to apply mutation testing based on the traversal of the abstract syntax tree (AST) is presented. Finally, an experiment is conducted to study the operator behaviour with different C++ programs, suggesting their usefulness in the creation of complete test suites. The analysis includes a Web service (WS) library, one of the domains where this technique can prove useful, considering its challenging testing phase and that C++ is still a reference language for critical distributed systems WS
Do Null-Type Mutation Operators Help Prevent Null-Type Faults?
The null-type is a major source of faults in Java programs, and its overuse
has a severe impact on software maintenance. Unfortunately traditional mutation
testing operators do not cover null-type faults by default, hence cannot be
used as a preventive measure. We address this problem by designing four new
mutation operators which model null-type faults explicitly. We show how these
mutation operators are capable of revealing the missing tests, and we
demonstrate that these mutation operators are useful in practice. For the
latter, we analyze the test suites of 15 open-source projects to describe the
trade-offs related to the adoption of these operators to strengthen the test
suite
Coverage-based quality metric of mutation operators for test suite improvement
The choice of mutation operators is a fundamental aspect in mutation testing to guide the tester to an effective test suite. Designing a set of mutation operators is subject to a trade-off between effectiveness and computational cost: a larger mutation population might uncover more faults, but will take longer to analyse. With the aim of resolving this trade-off, several authors have defined an assortment of metrics to determine the most valuable operators. In this work, we extend an existing quality metric by incorporating an additional source of data and coverage information and therefore investigate the extent to which mutants that are often covered but rarely killed can improve the evaluation of mutation operators for the refinement of the test suite. As a case study, we analyse C++ class-level operators based on the new coverage-based quality metric to assess whether the original metric is enhanced. The results when selecting the best-valued operators show that this metric has great potential to help the tester in finding effective mutation operators. In comparison with the metric from which it is derived, the use of coverage data allows to reduce the number of mutants but often loses fewer test cases and, in addition, retains those that seem hard to design
Software Testing Techniques Revisited for OWL Ontologies
Ontologies are an essential component of semantic knowledge bases and applications, and nowadays they are used in a plethora of domains. Despite the maturity of ontology languages, support tools and engineering techniques, the testing and validation of ontologies is a field which still lacks consolidated approaches and tools. This paper attempts at partly bridging that gap, taking a first step towards the extension of some traditional software testing techniques to ontologies expressed in a widely-used format. Mutation testing and coverage testing, revisited in the light of the peculiar features of the ontology language and structure, can can assist in designing better test suites to validate them, and overall help in the engineering and refinement of ontologies and software based on them
Towards learning and verifying invariants of cyber-physical systems by code mutation
Cyber-physical systems (CPS), which integrate algorithmic control with
physical processes, often consist of physically distributed components
communicating over a network. A malfunctioning or compromised component in such
a CPS can lead to costly consequences, especially in the context of public
infrastructure. In this short paper, we argue for the importance of
constructing invariants (or models) of the physical behaviour exhibited by CPS,
motivated by their applications to the control, monitoring, and attestation of
components. To achieve this despite the inherent complexity of CPS, we propose
a new technique for learning invariants that combines machine learning with
ideas from mutation testing. We present a preliminary study on a water
treatment system that suggests the efficacy of this approach, propose
strategies for establishing confidence in the correctness of invariants, then
summarise some research questions and the steps we are taking to investigate
them.Comment: Short paper accepted by the 21st International Symposium on Formal
Methods (FM 2016